Color by attribute #148

TeunHuijben · 2024-11-06T23:41:30Z

I closed #147, and started a new one on this topic:

This PR will implement the option to color the cells based on some attribute. The attribute could be any column added to the tracks.csv, or calculated features from the coordinates (think of velocity, direction, etc.)

Implemented:

Bugs:

inputText for dataURL no longer turns red when URL is false (the team behind SDS is working on this)

Example datasets:

Example of coloring based on z-coordinate:

vercel · 2024-11-06T23:41:34Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
intracktive	✅ Ready (Inspect)	Visit Preview	Dec 7, 2024 1:33am

…new version

…tion

…iewer into color-by-attribute

…f default attributes when fetching

…utes in conversion script

…n copying URL

…s, y-pos, quadrants, etc.

…ded)

TeunHuijben · 2024-12-07T01:34:32Z

Hi @andy-sweet and @aganders3,

This PR is a bit bigger and contains the feature to color cells based on provided attributes. In short, 1) extra columns in the tracks.csv file are saved in the Zarr Store, 2) the UI has a toggle and a dropdown menu where the user can select the feature used for coloring the cells. The Zarr Store encodes the names of the features and the type (categorical/continuous). The user can set the colormaps in the CONFIG, and there are separate colormaps for categorical/continuous features.

Considerations for review:

is this the best way to save/chunk the attributes into the Zarr Store?
for the larger datasets (zebrafish), I feel that coloring the cells according to a attribute that is loaded from the Zarr Store, make the viewer slower. Would you see any opportunity for speed-up? (the fastest way to would be to add the features to the points Zarr array ([x,y,z,attribute1,attribute2,...,x,y,z,attribute1,attribute2], but this would create a very large points array in the case of many attributes, especially since most of them are usually not used)
the "quadrants" coloring option is to show categorical features, not because it is useful

Thanks!

aganders3 · 2024-12-09T15:31:02Z

It looks like you're creating another ragged array in the top-level of the Zarr bundle as attributes that is roughly the same shape as points (well, num_attributes * 1/3 the size). Is that right? With the same chunking as points this should only be one extra request when the timepoint changes - I don't think that should be a huge deal but it is some overhead.

I agree packing it all into points is probably not optimal if there will be a lot of unused attributes. You may have the same issue in attributes if you have a lot of attributes saved. What about making attributes another group, and inside you could have separate arrays for each attribute?

TeunHuijben · 2024-12-11T18:39:57Z

Hi @aganders3, thanks for having a look! Yes, that is correct, attributes is a dense ragged array. The chunking is different from points though, where points is chunked per time, attributes is also chunked per attribute. In this way, only the requested attribute at the current timepoint is fetched (I hope I implemented this correctly).

What do you mean with "making attributes another group"? Do you mean to make a second zarr store with only the attributes? Or can we make a second group within the tracks_bundle.zarr store? I might be unfamiliar with the definitions or store/group/array. Because right now, attributes is an extra (the fifth) array within the same store.

aganders3 · 2024-12-11T19:27:14Z

The chunking is different from points though, where points is chunked per time, attributes is also chunked per attribute.

Ahh, I see. I missed this distinction and thought the attributes were still interleaved. Never mind my suggestion then, I think this achieves the same thing!

andy-sweet

I'm still making my way through this, but left lots of minor comments and one or two more important ones. Will try to take another look tomorrow.

Qualitatively, I noticed a minor performance hit, but not enough of one to block this feature which seems really useful!

andy-sweet · 2025-01-16T20:57:28Z

CONFIG.ts

@@ -11,16 +11,26 @@ const config = {
    // When opening the viewer, or refreshing the page, the viewer will revert to the following default dataset
    data:{
        // Default dataset URL (must be publically accessible)
-        default_dataset: "https://public.czbiohub.org/royerlab/zoo/Zebrafish/tracks_zebrafish_bundle.zarr/"
+        // default_dataset: "https://public.czbiohub.org/royerlab/zoo/Zebrafish/tracks_zebrafish_bundle.zarr/"
+        default_dataset: "https://public.czbiohub.org/royerlab/zoo/misc/tracks_drosophila_attributes_norm_bundle.zarr/"


Should this be the following?

Suggested change

default_dataset: "https://public.czbiohub.org/royerlab/zoo/misc/tracks_drosophila_attributes_norm_bundle.zarr/"

default_dataset: "https://public.czbiohub.org/royerlab/zoo/misc/tracks_zebrafish_displ_norm_bundle.zarr/"

andy-sweet · 2025-01-16T20:58:07Z

README.md

@@ -117,6 +117,13 @@ intracktive convert --csv_file path/to/tracks.csv --add_radius

 Or use `intracktive convert --help` for the documentation on the inputs and outputs

+**ToDo: explain how to add attributes**


Does this TODO still need to be done?

andy-sweet · 2025-01-16T21:00:09Z

src/components/App.tsx

@@ -223,17 +225,26 @@ export default function App() {
            const getPoints = async (time: number) => {
                console.debug("fetch points at time %d", time);
                const data = await trackManager.fetchPointsAtTime(time);
+                // console.log('data shape:', data.length, 'attributes shape:', attributes.length);


Maybe combine this with the debug log below? Or remove it.

andy-sweet · 2025-01-16T21:14:26Z

src/components/leftSidebar/DynamicDropdown.tsx

+    const [inputDropdownValue, setInputDropdownValue] = useState<string | undefined>(options[0]?.name);
+    const [value, setValue] = useState<AutocompleteValue<Option, false, false, false> | null>(options[0] || null);
+
+    // useEffect(() => {


Can this be removed? If not add a comment to the code to explain why it might be useful in the future.

andy-sweet · 2025-01-16T21:14:36Z

src/components/leftSidebar/DynamicDropdown.tsx

+            <DropdownMenu
+                open={open}
+                anchorEl={anchorEl}
+                // onClose={noop}


Is this still needed?

andy-sweet · 2025-01-16T21:49:33Z

src/lib/PointCanvas.ts

+        // const numPoints = positions.count / numberOfValuesPerPoint;
+        // console.log('numPoints in getAtt:',numPoints, positions.count, numberOfValuesPerPoint)
+        // console.log('positions:',positions)


Suggested change

// const numPoints = positions.count / numberOfValuesPerPoint;

// console.log('numPoints in getAtt:',numPoints, positions.count, numberOfValuesPerPoint)

// console.log('positions:',positions)

andy-sweet · 2025-01-16T21:50:08Z

src/lib/PointCanvas.ts

+                const quadrant = x + y * 2 + z * 4; //
+                attributeVector.push(quadrant); // color based on XY coordinates (4 groups)
+            } else {
+                attributeVector.push(1); // default to constant color if event type not recognized


Maybe an error should be logged in this case?

andy-sweet · 2025-01-16T21:53:19Z

src/lib/TrackManager.ts

@@ -153,6 +157,28 @@ export class TrackManager {
        return array;
    }

+    async fetchAttributessAtTime(timeIndex: number, attributeIndex: number): Promise<Float32Array> {


Suggested change

async fetchAttributessAtTime(timeIndex: number, attributeIndex: number): Promise<Float32Array> {

async fetchAttributesAtTime(timeIndex: number, attributeIndex: number): Promise<Float32Array> {

andy-sweet · 2025-01-16T22:10:33Z

src/components/leftSidebar/DynamicDropdown.tsx

+
+export const numberOfDefaultColorByOptions = DEFAULT_DROPDOWN_OPTIONS.length;
+// Initialize the mutable dropdown options with the default options
+export const dropDownOptions: Option[] = [...DEFAULT_DROPDOWN_OPTIONS];


I haven't made a drop-down in SDS, but the code here feels problematic on a first look because this is a mutable array of Options that is effectively used as React state.

For react state, the typical pattern is to just create a new array each time one is needed rather than mutate a single instance.

andy-sweet · 2025-01-16T22:11:16Z

src/components/leftSidebar/DynamicDropdown.tsx

+
+    return (
+        <div>
+            <InputDropdown


From the SDS docs, I'd expect to see DropDown here instead of InputDropdown and DropdownMenu.

But I've never actually used any of those, so probably ignore?

andy-sweet

I think the zarr storage format and fetching (when needed) is as reasonable as prior choices we've made, so I don't have any feedback related to that or the Python conversion code/tools.

My main comment is about the how you handle the state related to the attribute drop-down box value changing. I previously left some related comments, but I made those a bit more tangible by trying some changes that I think simplify things and also fix some bugs: https://github.com/royerlab/inTRACKtive/compare/color-by-attribute...andy-sweet/refactor-attr-options?expand=1#diff-8e6c274c4241821e809d16fcd06bc109ba41bb7a0583f9ad766ed92bb8230671R8

I don't think you should blindly merge those changes, but I think the approach there is easier to follow and importantly does not make the library code depend on definitions from React components.

andy-sweet · 2025-01-17T20:51:24Z

src/components/leftSidebar/DynamicDropdown.tsx

+    { name: "y-position", label: 2, type: "continuous", action: "calculate", numCategorical: undefined },
+    { name: "z-position", label: 3, type: "continuous", action: "calculate", numCategorical: undefined },
+    { name: "sign(x-pos)", label: 4, type: "categorical", action: "calculate", numCategorical: 2 },
+    { name: "quadrants", label: 4, type: "categorical", action: "calculate", numCategorical: 8 },


I'm assuming these two options should not have the same label.

Suggested change

{ name: "quadrants", label: 4, type: "categorical", action: "calculate", numCategorical: 8 },

{ name: "quadrants", label: 5, type: "categorical", action: "calculate", numCategorical: 8 },

andy-sweet · 2025-01-17T20:58:11Z

src/lib/PointCanvas.ts

@@ -24,13 +26,17 @@ import { Track } from "@/lib/three/Track";
 import { PointSelector, PointSelectionMode } from "@/lib/PointSelector";
 import { ViewerState } from "./ViewerState";
 import { numberOfValuesPerPoint } from "./TrackManager";
+import { Option, dropDownOptions } from "@/components/leftSidebar/DynamicDropdown";


As a rule, we tried to avoid make the code in lib depend on definitions in components, so we can define the core logic of the library and rendering independently of React.

TeunHuijben added 4 commits November 5, 2024 18:07

initial commit with dropdown window and coloring based on position

2c35c08

lint fixes

154276a

interpolate between two opposite colors + added extra test features

1276f57

replaced dropdown from mui to czi-sds/components

3a8a6a7

changed Button and InputText czi-sds/components elements, because of …

90166f0

…new version

vercel bot deployed to Preview – intracktive November 7, 2024 17:56 View deployment

TeunHuijben added 2 commits November 7, 2024 14:56

categorical/continuous colormaps

f5601d4

fixed czi-sds/components after new version

9866c1b

vercel bot deployed to Preview – intracktive November 7, 2024 23:42 View deployment

attributes are saved in zarr and loaded when requested in the applica…

9bd4b0f

…tion

vercel bot deployed to Preview – intracktive November 11, 2024 22:10 View deployment

fixed width of dropdownMenu

1b2c730

TeunHuijben added the enhancement New feature or request label Nov 15, 2024

Merge branch 'main' into color-by-attribute

254545f

vercel bot had a problem deploying to Preview – intracktive November 21, 2024 00:14 Failure

TeunHuijben added 3 commits November 20, 2024 16:15

Merge branch 'color-by-attribute' of github.com:royerlab/points-web-v…

1c49f2d

…iewer into color-by-attribute

fixed mistakes in convert after merge with main

815ca92

replaced HTTPStore by .attrs.asObject

7a4c81e

vercel bot deployed to Preview – intracktive November 21, 2024 00:27 View deployment

updated conversion test

9c492cb

vercel bot deployed to Preview – intracktive November 21, 2024 00:59 View deployment

TeunHuijben added 2 commits November 20, 2024 17:34

reset dropdown after loading new dataset + removed hardcoded number o…

f184bd1

…f default attributes when fetching

unified colormaps, toggle for colorBy, second colormap legend

601ac5f

vercel bot deployed to Preview – intracktive November 22, 2024 00:39 View deployment

TeunHuijben added 3 commits November 22, 2024 14:29

added colorBy colorbar overlay + added flag for pre-normalized attrib…

6f4b851

…utes in conversion script

changed default dataset(s) + color legend only visible when relevant

836b585

changed text on colormap legend overlays

15ae251

vercel bot deployed to Preview – intracktive November 22, 2024 23:17 View deployment

demo ready PR for SI videos (removed some default attributes)

f9fa997

vercel bot deployed to Preview – intracktive December 3, 2024 22:15 View deployment

added conversion tests + conversion detects categorical/continuous

2d5c94f

vercel bot deployed to Preview – intracktive December 5, 2024 17:36 View deployment

trying to fix lint issues

76d4dae

vercel bot deployed to Preview – intracktive December 5, 2024 17:53 View deployment

fixed wrong dataset names

b70421b

vercel bot deployed to Preview – intracktive December 6, 2024 00:11 View deployment

TeunHuijben added 3 commits December 5, 2024 16:32

toggle colorBy to off when new dataset is loaded

a0f18cc

Merge branch 'main' into color-by-attribute

8835848

colorBy toggle only visible when Zarr Store has attributes

8252941

vercel bot deployed to Preview – intracktive December 6, 2024 22:48 View deployment

TeunHuijben added 3 commits December 6, 2024 14:57

saved colorBy toggle and field in ViewerState, so it is preserved whe…

f555d56

…n copying URL

config option to show/not show the default dropdown options like x-po…

597b5db

…s, y-pos, quadrants, etc.

config option to disable coloring cells (even if attributes are provi…

0eb31b4

…ded)

vercel bot deployed to Preview – intracktive December 7, 2024 01:06 View deployment

changed example default dataset (includes attributes)

16c4e55

vercel bot deployed to Preview – intracktive December 7, 2024 01:33 View deployment

TeunHuijben marked this pull request as ready for review December 9, 2024 01:58

TeunHuijben requested review from andy-sweet and aganders3 December 9, 2024 01:58

andy-sweet reviewed Jan 16, 2025

View reviewed changes

andy-sweet reviewed Jan 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Color by attribute #148

Color by attribute #148

TeunHuijben commented Nov 6, 2024 •

edited

Loading

vercel bot commented Nov 6, 2024 •

edited

Loading

TeunHuijben commented Dec 7, 2024 •

edited

Loading

aganders3 commented Dec 9, 2024

TeunHuijben commented Dec 11, 2024

aganders3 commented Dec 11, 2024

andy-sweet left a comment

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet Jan 16, 2025

andy-sweet left a comment

andy-sweet Jan 17, 2025

andy-sweet Jan 17, 2025

	default_dataset: "https://public.czbiohub.org/royerlab/zoo/misc/tracks_drosophila_attributes_norm_bundle.zarr/"
	default_dataset: "https://public.czbiohub.org/royerlab/zoo/misc/tracks_zebrafish_displ_norm_bundle.zarr/"

		@@ -117,6 +117,13 @@ intracktive convert --csv_file path/to/tracks.csv --add_radius

		Or use `intracktive convert --help` for the documentation on the inputs and outputs

		ToDo: explain how to add attributes

	// const numPoints = positions.count / numberOfValuesPerPoint;
	// console.log('numPoints in getAtt:',numPoints, positions.count, numberOfValuesPerPoint)
	// console.log('positions:',positions)

	async fetchAttributessAtTime(timeIndex: number, attributeIndex: number): Promise<Float32Array> {
	async fetchAttributesAtTime(timeIndex: number, attributeIndex: number): Promise<Float32Array> {

	{ name: "quadrants", label: 4, type: "categorical", action: "calculate", numCategorical: 8 },
	{ name: "quadrants", label: 5, type: "categorical", action: "calculate", numCategorical: 8 },

Color by attribute #148

Are you sure you want to change the base?

Color by attribute #148

Conversation

TeunHuijben commented Nov 6, 2024 • edited Loading

vercel bot commented Nov 6, 2024 • edited Loading

TeunHuijben commented Dec 7, 2024 • edited Loading

aganders3 commented Dec 9, 2024

TeunHuijben commented Dec 11, 2024

aganders3 commented Dec 11, 2024

andy-sweet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andy-sweet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TeunHuijben commented Nov 6, 2024 •

edited

Loading

vercel bot commented Nov 6, 2024 •

edited

Loading

TeunHuijben commented Dec 7, 2024 •

edited

Loading